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A METHOD AND APPARATUS FOR IMAGE SENSING USING AN 
INTEGRATED CIRCUIT-BASED COMPOUND EYE 

BACKGROUND 
5 1. Field 

An embodiment of the present invention relates to the field of imaging 
devices and methods and, more particularly, to an integrated circuit-based, 
artificial compound eye. 

2. Discussion of Related Art 

10 The market for image sensing devices is growing rapidly as new 

applications for such devices continue to arise. Current applications include 
camcorders, facsimile machines, digital cameras, toys and personal computer 
(PC) cameras, for example. Cameras and/or other types of image sensing 
devices are also expected to soon be included in items such as cars, wireless 

15 telephones, and personal digital assistants, for example. 

For many applications, there is a drive to continue to make components, 
including image sensing components, smaller. In some cases, however, 
currently available technology limits the extent to which these devices may be 
scaled. Current cameras, such as PC cameras, for example, typically include a 

20 single lens to focus light energy onto photosensitive circuitry. Such single lens 
systems do not scale well to small sizes and weights, however, due to issues 
such as lens curvature and focal length, for example, and thus, their utility may 
be limited for current and/or future applications where very small and/or 
lightweight image sensing components are desired. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and not limitation in 
the figures of the accompanying drawings in which like references indicate 
similar elements, and in which: 

Figure 1 illustrates a cross-sectional view of an integrated circuit-based 
compound eye of one embodiment. 

Figure 2 illustrates a cross-sectional view of an exemplary integrated 
circuit-based compound eye of one embodiment including a microlens array. 

Figure 3 illustrates a perspective view of the light directing member of the 
integrated circuit-based compound eye of Figure 2. 

Figure 4 illustrates a cross-sectional view of a conventional single-lens- 
based system. 

Figure 5 illustrates an overhead view of an exemplary lenslet array, 
associated photodetector elements and exemplary compound eye wiring that 
may be used for the embodiment of Figure 2 . 

Figure 6 illustrates a cross-sectional view of an exemplary integrated 
circuit-based compound eye of another embodiment including a light pipe 
bundle. 

Figure 7 illustrates an overhead view of a light pipe array that may be 
used for the embodiment of Figure 6. 

Figur 8 shows additional detail of some exemplary light pipes that may 
be used for the embodiment of Figur 6. 



Figur 9 illustrates a cross-sectional view of an exemplary integrated 
circuit-based compound eye of another embodiment including micromachined 
light directing elements. 

Figure 10 illustrates an overhead view of an exemplary micromachined 
light directing element of the embodiment of Figure 9. 

Figure 11 illustrates a cross-sectional view of an exemplary integrated 
circuit-based compound eye of another embodiment including micromachined 
light scanning elements with integrated photodetector elements. 

Figure 12 illustrates an overhead view of a micromachined light scanning 
element of the embodiment shown in Figure 1 1 . 

Figure 13 is a flow diagram illustrating the method of one embodiment for 
image sensing using an integrated circuit-based compound eye. 

DETAILED DESCRIPTION 

A method and apparatus for image sensing using an integrated circuit- 
based compound eye is described. In the following description, particular types 
of integrated circuits, lens configurations and materials, for example are 
described for purposes of illustration. It will be appreciated, however, that other 
embodiments are applicable to other types of integrated circuits, and to light 
directing elements and/or image sensors configured in another manner. 

For one embodiment, referring to Figure 1 , an apparatus comprises a 
plurality of photodetector elements 105 (also referred to herein as pixels, light 
receptor, photoreceptor and/or optoelectronic elements) disposed on a substrate 



110, such as a semiconductor substrate. The apparatus also includes a light 
directing member 115, also referred to herein as an exposure determining 
member, including a plurality of light directing elements 120 (only a few of which 
are indicated in Figure 1). 
5 For this embodiment, at least some of the light directing elements are 

provided to individually direct light energy from one or more sources 125 onto 
one or more of the photodetector elements 105. The light directing member 115 

«... provides the primary light directing means to direct light energy onto the 

p 

P photodetector elements 105, i.e. there are no other lenses or light directing 

pi 

W 10 members provided between the light source(s) 1 25 and the light directing 

:)pj3a 

Wj member 115. The photodetectors 105 are electrically coupled together such that 

111 

f t an image corresponding with the source 125 may be synthesized at output 

is as ^ 

3 

n\ 

y s circuitry. 

P For one embodiment, the light directing member 115 includes an array of 

15 lenslets, also referred to as a microlens array. For another embodiment, the light 
directing member 115 includes a plurality of light pipes bundled together. For 
still another embodiment, the light directing member 115 includes 
micromachined elements that are movable to determine the direction from which 
photons are provided to underlying photodetector elements. 
20 In accordance with an alternative embodiment, the exposure determining 

members are micromachined light scanning elements that each include a 
photodetector element. The micromachined light scanning elements are 
provided on a single substrate and controlled by a micromachine control circuit to 
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direct at least some of the micromachined scanning elements to receive light 
from one or more sources. 

Details of these and other embodiments are provided in the following 
description. 

5 Referring to Figure 2, a high level cross-sectional diagram of an' 

integrated circuit-based compound eye 200 of one embodiment is shown. The 
artificial compound eye 200 includes a plurality of photodetector elements 205 
provided on a semiconductor or other type of substrate 210 that may be part of, 
ft for example, an integrated circuit (IC) device. While only four representative 

5 

II) 10 photodetector elements 205a-d are shown in Figure 2, it will be appreciated that 

m 

4° the number of photodetector elements 205 on the IC 210 may be much larger. 

Ill ■ 

V- The photodetector elements 205 (also referred to as optoelectronic 

P elements) may be arranged, for example, in a two-dimensional, regular or 
i « irregular array to provide a photosensitive region 206 on the IC 21 0. 
jp 15 For one embodiment, the photodetector elements 205 are complementary 

metal oxide semiconductor-based (CMOS-based) photodetector elements. For 
another embodiment, charge-coupled devices (CCDs) may be used to provide 
the photodetector elements 205. For still another embodiment, mixed 
CMOS/CCD technology may be used to provide the photodetector elements 205. 
20 Other types of photodetector elements are within the scope of various 
embodiments. 

Each of the photodetector elements 205 is responsive to incident photons 
from a light source, such as the light source 225, to provide an electrical signal 
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indicative of the energy or intensity of the light spectrum to which it is sensitive. 
Other circuitry (not shown) either on the IC 210 or coupled to the IC 210 may 
then convert the electrical signal in a manner well-known to those of ordinary skill 
in the art to provide an associated digital signal. 
5 For the artificial compound eye 200 of Figure 2, a light directing member 

215 is disposed between a light source 225 and the photodetectors 205. The 
light directing member 215 for the embodiment of Figure 2 is a compound light 
directing member that includes a lenslet array (also referred to as a microlens or 
Jjj microlenticular array) as shown in perspective view in Figure 3. With continuing 

if* I 

jpj 10 reference to Figure 2, the light directing member 215 is referred to as a 

m compound light directing member because the lenslet array 215 includes a 

Ml 

* plurality of lenslets 230 that are each capable of acting as a light directing 

R| element. Exemplary lenslets 230a-c are shown in Figure 2, but it will be 

fl appreciated that the lenslet array 21 5 may include many more such lenslets, 

w 

15 including more lenslets than shown in Figure 3. 

The lenslets 230 may be arranged in any sort of pattern on the light 
directing member 215. For some embodiments, the arrangement, spacing, 
sizes, etc. of the lenslets 230 in the lenslet array 215 may depend on 
considerations such as, for example, the method and/or materials used to 

20 manufacture the lenslet array 215, available area, characteristics of the 

underlying photodetectors 205, the particular application for the compound eye 
200, desired characteristics (e.g. resolution, sensitivity, allowable noise, desired 
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dynamic range) of the compound eye 200, etc. Other factors may also be 
considered. 

The lenslet array 215 may be manufactured directly on an integrated 
circuit (IC) die or other substrate 210 as part of the IC fabrication process. 
5 Alternatively, the lenslet array 21 5 may be manufactured separately from the 
substrate 210 and then optically coupled with the substrate 210 during a later 
manufacturing step such as, for example, packaging. For the second case, the 
lenslet array 215 may be, for example, embossed on a plastic or other type of 
cover or window (not shown) that is provided adjacent to the IC 210. 

1 0 There are several different, known manufacturing techniques for suitable 

lenslet arrays. Some such techniques are provided, for example in a publication 
entitled, "Fabrication of Refractive Microlens Arrays by Visible Irradiation of 
Acrylic Monomers: Influence of Photonic Parameters," by C. Croutxe-Barghorn 
et al., Eur. Phys. J. AP 13, 31-37 (2001), the text of which, at the time of this 

1 5 application, may be found at 

www.edpsciences.ora/articles/epiap/pdf/2001/01/ap0122.pdf . Alternative lenslet 
array manufacturing techniques and/or materials not mentioned in this 
publication may also be used to manufacture the lenslet array 215 for various 
embodiments. 

20 The lenslet array 21 5 is positioned with respect to the plurality of 

photodetector elements 205 such that at least some of the lenslets 230 focus 
light energy from the light source 225 onto one or more of the photodetectors 
205. For example, light received at the lenslet 230c may be provided either to 




photodetector element 205c or to photodetector element 205d depending the 
incident angle of the light. 

Further, for one embodiment, the distance 227 between the light directing 
member 215 and a surface of the substrate 210 may be less than or equal to 
5 .0001 meter, for example, while the distance 229 between the light directing 
member 215 and the light source of interest 225 may be greater than or equal to 
.1 meter. It will be appreciated that different separations between the light 
directing member 215 and the light source of interest 225 and/or between the 
light directing member 215 and the substrate 210 may apply to different 

10 embodiments. 

With continuing reference to Figure 2, the multiple lenslets in the lenslet 
array 215 provide the primary mechanism to focus light from the light source 225 
onto the photodetector elements 205. In other words, for this embodiment, there 
is no additional lens or other type of light directing member provided between the 

1 5 lenslet array 215 and the light source 225. Because the lenslet array 21 5 is used 
to focus light onto the photodetector elements 205, mapping an output of each of 
the photodetector elements 205 to a corresponding point in an output image may 
not be as straightforward as for a single lens-based image sensor. 

For the compound eye 200 of Figure 2, in order to be able to synthesize 

20 an output image that corresponds to the light source 225 or to another image 
source in response to photons directed onto the photodetector elements 205 by 
the lenslet array 215, the photodetector elements 205 are interconnected 
according to a compound eye wiring pattern 235 including image integration 




circuitry 236, as described in more detail below. In other words, whereas 
conventional cameras use single large lenses to integrate light from point 
sources onto points on underlying film or retinas, integration of light energy using 
the compound eye 200 takes place via the wiring and circuits interconnecting 
5 and interpreting outputs of the photodetector elements 205. 

This is in contrast to some conventional applications for microlenses. 
Referring to Figure 4, as discussed in, for example, a publication entitled, 
"Anatomy of a Digital Camera: Image Sensors," by Sally Wiener Grotta, Extreme 

?! Tech, June 1 0, 2001 , which can, at the time of filing this application, be found at 

Q 

£j 10 ( http://www.extremetech.com/print article/0.3428.a%253D2036.00.asp ). In 

■ 11 

in some currently available digital cameras, for example, microlenses 405 may be 

Si 

, grown or otherwise placed above individual CCDs or CMOS sensor devices 41 0 

fU (only a representative few of each of the microlenses 405 and sensor devices 

2 : 

410 are shown in Figure 4). These microlenses 405 are used to increase the 

Li. 

15 angular response, and thus, photosensitivity of such sensor devices 410. The 
microlenses 405, however, receive incident light via a single lens 412 that is 
situated between a light source 415 and the microlenses 405 as shown in Figure 
4. For such a single lens-based system, it is not necessary or useful to 
interconnect sensor devices 410 using a compound eye wiring pattern similar to 

20 that described above. This is because the physical locations of the sensors 41 0 
more directly correspond to the physical locations of pixels in a resultant image. 

While such single lens-based systems may provide a more 
straightforward relationship between the relative physical location of a sensor 
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and that of a corresponding element of a resultant image, the scalability and 
other limitations of such single lens-based systems may prevent them from being 
practical for applications, for example, where very small weight and/or form 
factors are desired as discussed above. 

Referring again to Figure 2, according to the compound eye wiring pattern 
235, photodetectors 205 that are physically adjacent to each other on the 
substrate 210 may not necessarily provide portions of a captured image that are 
physically adjacent to each other. For purposes of illustration, reference is made 
to Figure 5, which provides an overhead view of exemplary lenslets 230x and 
230y in the lenslet array 215 and a plurality of photodetectors 205 underlying 
each of the lenslets 230x and 230y. For this exemplary embodiment, the sizes 
of the lenslets 230x and 230y in relation to the photodetectors 205 are such that 
a single lenslet 230 may overlie (in relation to the light source 225) multiple 
photodetectors 205 as shown. 

With the arrangement shown in Figure 5, for example, light from a 
particular angle, direction and distance may be directed by individual lenslets 
230x and 230y onto each of the corresponding photodetectors 205 marked with 
an X. As such, each of these photodetectors 205 may be wired together in 
accordance with the compound eye wiring pattern 235 even though they are not 
physically adjacent to each other. Thus, in accordance with the compound eye 
wiring pattern 235, each photodetector element 205 is locally connected to many 
neighboring photodetector elements as well as being globally connected to many 
corresponding photodetector elements under different lenslets. 



10 




The image integration circuitry 236 of one embodiment includes hybrid 
analog/digital circuitry such as artificial retina circuitry developed by Carver Mead 
at the California Institute of Technology. This hybrid digital/analog circuitry acts 
to interconnect and interpret outputs of the photodetector elements 205 to 
5 provide an integrated image at output circuitry 207. The image integration 
circuitry 236 may provide capabilities such as lateral inhibition for edge 
enhancement, and/or time differentiation for movement detection, for example. 
Alternatively, these capabilities may be provided in image processing circuitry 
O (not shown). Other types of image-related capabilities may also be provided by 

q 

III 10 image integration circuitry 236. Examples of such circuitry are described, for 

CO 

jfl example, in a paper entitled, "A Biologically Motivated Imager and Motion 

in 

m 

Detector With Pixel Level Image Processing," by A. Moini and A. Bouzerdoum, 

3 

III Australian Microelectronics Conference, 29 Sept. - 3 Nov., 1997, Melbourne, the 

H 

$ text of which can, at the time of this filing, be found at 

.•? '•• -~ 

H 1 5 http://www.eleceng.adelaide.edu.au/Groups/GAAS/Bugeye/pub_bug/micro97.pdf 
Other types of image integration circuitry are within the scope of various 
embodiments. 

For one embodiment, image integration circuitry 236 and any other image 
processing circuitry included in the compound eye 200 are designed such that, 
20 for example, edge enhancement, motion detection and/or pattern recognition can 
be parallelized and take place concurrently with photoreception. 

Referring back to Figure 2, the compound eye wiring pattern 235 for the 
entire photosensitive region 206 of the IC 210 may be determined using a 
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number of different approaches. For one embodiment, the compound eye wiring 
pattern 235 is determined by applying a light source to the compound eye 200 
and determining which of the photodetectors 205 is responsive to the incident 
light. The position of the light source is then varied according to the application 
5 for the compound eye 200 until the compound eye wiring pattern 235 is 
identified. 

Alternatively, the compound eye wiring pattern 235 may be determined 
using mathematical equations similar to equations used to determine the 

:i ; 

%\ operation of a compound eye of, for example, an insect. Examples of use of 
}Ji 10 such equations are described in, for example a paper entitled, "Silicon Retina 

as 

jjj with Adaptive Filtering Properties, " by Shih Chii Liu, NIPS Proceedings, Nov. 

sa 

Mi 

s 1 997, the text of which can presently be located at 

I* 

pi http://www.pcmp.caltech.edu/anaprose/shih/. Other types of mathematical 

isss 

J~ approaches for determining the compound eye wiring pattern 235 are within the 

is? 

u 15 scope of various embodiments. 

The compound eye wiring pattern 235 will depend on many factors in 
addition to the configuration of the lenslet array 115 such as whether the 
designer wishes, for example, to implement lateral inhibition, whether color or 
grayscale images are desired, whether movement detection is desired and, if so, 
20 what kinds of movement detection are desired, whether the circuit behavior 
under low light conditions is desired to behave differently than under high light 
conditions, whether the circuitry needs to consume minimal power, and so on. 
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Where the lenslet array is not directly fabricated on the IC 210, positioning 
the lenslet array 215 with respect to the photosensitive region 206 on the IC 210 
including the photodetectors 205 may be accomplished in different ways 
depending on the relative sizes of the lenslets 230, the lenslet array 215, the 
photosensitive region 206, the compound eye wiring pattern 235, and other 
factors. For one embodiment, for example, it is only necessary to grossly align 
the lenslet array 215 over the photosensitive region 206. The compound eye 
wiring pattern 235 then determines the relative relationship between each of the 
photodetectors 205 and picture elements of a resulting, corresponding image. 

For another embodiment, more precise alignment may be beneficial. For 
such embodiments, it may be possible to use conventional alignment tools to 
precisely align particular lenslet(s) 230 over particular photodetector(s) 205. 
Other approaches to aligning the lenslet array 215 with respect to the IC 210 are 
within the scope of various embodiments. 

With continuing reference to Figure 2, for one embodiment, the 
photodetector elements 205 are electrically coupled to integrated and/or off-chip 
output circuitry 207 such that an image captured by the photodetector elements 
205 corresponding to light source 225 may be provided at outputs of the output 
circuitry 207. The output circuitry 207 may be coupled to, for example, a display 
device. Alternatively, output circuitry 207 may include a memory device such as 
a flash memory to store an image captured by the photoreceptor elements 205 in 
response to light energy. It will be appreciated that other types of circuitry and/or 
elements such as, for example, a color filter array, an analog to digital converter, 
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shutter control circuitry, and/or additional image processing circuitry that may 
include such capabilities as, for example, demosaicing, resolution interpolation, 
sub-sampling, etc., may also be included, either on the IC 210 itself or may be 
off-chip and electrically coupled to the IC 210. 
5 In operation, photons from point light sources such as the light source(s) 

225 corresponding to the image to be captured strike lenslets 230 of the lenslet 
array 215. At least some of the lenslets 230 direct the photons to be received by 
one or more associated photodetector elements 205. The photodetector 
elements 205 that receive the photons are responsive to the photons to produce 

1 0 an electrical signal at an output of the photodetector element 205, wherein the 
magnitude of the electrical signal is related to the energy of the incident photons. 

Image integration circuitry 236 and other image processing circuitry (not 
shown) may convert these electrical signals to digital signals and provide other 
image processing capabilities. The compound eye wiring pattern 235 

15 determines a relationship between the signals provided at outputs of the 

photodetectors 205 and a resulting image that is provided at outputs of output 
circuitry 207. 

Where the compound eye 200 is a sensor for a PC camera, for example, 
the output image may be in the form of a digital moving image. Alternatively, 
20 where the compound eye 200 is a sensor for a digital still camera, the output 

image may be in the form of a digital still image. The compound eye 200 may be 
used for many other types of applications with many other types of 
corresponding output images. For example, the compound eye 200 may be 

14 
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used for tracking objects, controlling appliances or machinery, monitoring 
processes, etc. 

Referring now to Figure 6, a cross-section of a compound eye 600 in 
accordance with another embodiment is shown. For the compound eye 600, 
5 photodetector elements 605 are disposed on an integrated circuit (IC) or other 
substrate 610 as described above in reference to Figure 2 to provide a 
photosensitive region 606 on the IC 610. The photodetector elements 605 may 
be similar in function, operation, arrangement, and/or construction to the 
O photodetector elements 205 described in reference to Figure 2. 

if |] _ 

i;\ 10 For the exemplary compound eye 600, as for the compound eye 200, a 

111 compound light directing member 615 is disposed between one or more light 

if! 

* sources of interest 625 and photodetector elements 605. In contrast to the 

fU compound eye 200 of Figure 2, for the compound eye 600 of Figure 6, the light 

Li. 

f} h directing member 615 includes a plurality of light pipes 630 bundled together to 
! ' :s 15 provide a light pipe array 615, also referred to herein as a light pipe bundle. 

Figure 7 is an overhead view showing an exemplary arrangement of such 
a light pipe array 615, although a different number of light pipes and/or a 
different light pipe arrangement may be used for various embodiments. For 
some embodiments, the arrangement, spacing, diameters, etc of the light pipes 
20 630 may depend on factors similar to those considered for the lenslet array 215. 

Referring back to Figure 6, each of the light pipes 630, like the lenslets 
230 of Figur 2, acts as a light directing element, wherein at least some of the 
light pipes 630 direct light to be received by one or more corresponding 

15 
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photodetector elements 605. Exemplary light pipes 630a-630d are shown in 
Figure 6, but it will be appreciated that the light pipe bundle 615 may include 
many more such light pipes 630. 

Figure 8 shows a few exemplary light pipes 630 in more detail. As shown 
5 in Figure 8, the light pipes 630, also referred to as optical fibers, each comprise 
a higher refractive index region 805 and a lower refractive index region 810. 
Because the fibers 630 are clad with material of higher refractive index than their 
cores 810, photons incident on one end of a fiber 630 are directed down the fiber 
630 and to underlying photodetectors 605. If external ends 815 of the light pipes 

Tar 

ro 

P 10 630 are planar, photons entering at an angle normal to the plane of the external 

;«bs 

|I5 end 815 are preferentially conducted down the light pipe 630. 

in 

■ The higher refractive index region 805 causes a portion of the light that 

!:! 

W enters the light pipe 630 at an angle to be reflected as shown. Thus, only a 

|=* 

^ portion of the light that enters each light pipe 630 is directed to an underlying or 
15 otherwise associated photodetector 605. So, because of their construction, light 
pipes 630 may have a smaller angular response than the lenslets 230 described 
above, i.e. light that is passed through the light pipes 630 may need to be closer 
to perpendicular to a surface 815 of the light pipe 630 that initially receives the 
light. The above factors may result in reduced efficiency of the compound eye 
20 600 as compared to the compound eye 200 of Figure 2 in terms of light capture. 

For one embodiment, however, the compound eye 600 of Figure 6 may 
provide more flexibility than the compound eye 200 of Figur 2 in an optical 
sense. This is because the optics of the lenslet-based eye 200 places more 
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constraints on the underlying integration circuitry (i.e. compound eye wiring 
pattern and image integration circuitry) as compared to the compound eye 600 of 
Figure 6. For the compound eye 200 of Figure 2 of one embodiment, for 
example, there is a one lenslet 230 to many photodetector element 205 mapping 
5 while for the compound eye 600 of Figure 6, the mapping of light pipes 630 to 
photodetector elements 605 may be closer to one-to-one. 

Also, for the light pipes 630, for one embodiment, referring to Figures 6 
and 8, the external surfaces 815 of the light pipes 630, i.e. the surfaces that 
initially receive photons from one or more light sources of interest 625, can be 

pi 

m 10 cut or otherwise altered to vary the angle of the external surface. The angles of 

> 
:S3S 

|h the external surfaces 815 determine at which point sources of light and/or at 

in 

* which combination of point sources each particular light pipe 630 is aimed. 

W Referring to Figure 6, for example, as shown in the exploded view 635, 

l« 

J2( the light pipe 630a may have an external surface 640 cut at the angle shown to 

a . 

15 preferentially receive light from one direction while the light pipe 630d may have 
an external surface 645 cut at the angle shown in exploded view 650 to 
preferentially receive light from another direction. 

Thus, the external surfaces of the light pipes 630 may all be at the same 
angle with respect to each other or they may have different angles with respect 

20 to each other depending on the particular application and desired characteristics 
of the resultant compound eye 600. In this manner, the compound eye 600 may 
provide improved flexibility in terms of designing the spatial sensitivity of the eye 
600. 

17 
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The light pipe bundle 615 of one embodiment may be manufactured in a 
variety of different ways in accordance with known technology. Some exemplary 
light pipe bundles that may be suitable to provide the light pipe bundle 615 are 
provided by Schott Fiber Optics of Southbridge, MA, a division of Schott 
5 Corporation of Yonkers, NY. In particular, light pipe bundles similar to Schott's 
fused fiber optic faceplates may be used to provide the light pipe bundle 615. 
Alternative approaches to manufacturing a suitable light pipe bundle are within 
the scope of various embodiments. 

O With continuing reference to Figure 6, the compound light directing 

P 

5! 1 0 member 61 5 may be optically coupled with the photodetector elements 605 by 

w 

33 

j|J including the compound light directing member 615 as an optical window in a 

! ! package for the IC device 610, for example. For another embodiment, the 

I s * 

jy compound light directing member 615 may be optically coupled with at least 

\ij some of the photodetector elements 605 in another manner. For one 

.5 S3, 

u 

H 15 embodiment, the light directing member 615 is only grossly aligned with the 
photosensitive region provided by photodetector elements 605 and the 
relationship of the photodetector elements 605 to the light pipe array 615 is 
determined after manufacturing. For another embodiment, more precise 
alignment may be desirable in which case, commercially available alignment 
20 machinery may be used. 

For one embodiment, a distance 627 between the light directing member 
615 and a surface of the IC 610 may be less than or equal to .0001 meter, for 
example, while a distance between the light directing member and a light source 



18 




of interest may be greater than or equal to .1 meter. It will be appreciated that 
either of the above distances may be different for other embodiments. For 
example, for one embodiment, the light directing member 615 may be placed in 
direct contact with a surface of the substrate 610 including the photodetectors 
5 605. 

As for the compound eye 200, to integrate an image corresponding to the 
one or more light sources of interest 625 at output circuitry 607, outputs of the 
photodetector elements 605 of the compound eye 600 are wired together using a 
compound eye wiring pattern 635 and image integration circuitry 636. According 

1 0 to the compound eye wiring pattern 635, photodetectors 605 that are physically 
adjacent to each other on the substrate 610 may not necessarily provide portions 
of a captured image that are physically adjacent to each other. 

The image integration circuitry 636 of one embodiment may be similar in 
function, capabilities and/or construction to the image integration circuitry 236 of 

15 Figure 2. 

Due to the different mappings of light directing elements 630 to 
photoreceptors 605 for the compound eye 600 as compared to the compound 
eye 200, the compound eye wiring pattern 635 may differ from the compound 
eye wiring pattern 235. The compound eye wiring pattern 635, however, may be 
20 determined in a similar manner to the compound eye wiring pattern 235 to 
provide an integrated output image at output circuitry 607 that corresponds to 
one or more light sources of interest 625. 
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It should be noted that, whatever mapping maps point sources of light in 
the environment 635, 650 of Figure 6 to particular receptors 605, the mapping 
could be studied and more fully characterized after manufacture by moving point 
sources of light across the "visual field" of the device and recording its output 
5 607. This kind of post-hoc mapping, however, is not necessarily useful for 
designs for which circuits are based on particular geometric relationships such 
as lateral inhibition. For these cases, the mapping should preferably be 
implemented in the manufacturing process. 

In operation, photons from point light source(s) 625 corresponding to the 

10 image to be captured are received at at least some of the light pipes 630. At 
least some of the light pipes 630 direct photons to be received by one or more 
corresponding photodetector elements 605, which are each responsive to the 
photons to produce an electrical signal at an output that is related in magnitude 
to the energy of the incident photon. 

15 Image integration circuitry 636 and, for some embodiments, other image 

processing circuitry (not shown) may convert these electrical signals to digital 
signals and provide other image processing capabilities. The compound eye 
wiring pattern 635 determines a relationship between the signals provided at 
outputs of the photodetectors 605 and a resulting image provided at outputs of 

20 output circuitry 607. 

As for the compound eye 200, the form of the output image may depend 
on the particular application for the compound eye 600. Further, the compound 
eye 600 may be used for any number of image sensing applications including, 

20 



for example, tracking objects, controlling machinery or appliances, camera 
applications, etc. 

Figure 9 is a cross-sectional view of a compound eye 900 of another 
embodiment. For the compound eye 900, photodetectors 905 are provided on a 
semiconductor or other type of substrate 910. The photodetectors 905 and/or 
the substrate 910 may be similar in construction, characteristics and/or operation 
to the photodetectors 205 and/or the photodetectors 605 of Figures 2 and 6, 
respectively. 

The compound eye 900 also includes light directing elements 930. For 
the embodiment shown in Figure 9, the light directing elements 930 comprise 
micromachined light directing elements, each of which has an opening 1007 as 
shown in Figure 10 to allow light to pass through and be received by underlying 
photodetector(s) 905. While the opening 1007 is round in Figure 10, it will be 
appreciated that the opening may be a different shape for other embodiments. 

Only a representative few micromachined elements 930a-d are shown in 
Figure 9, however, it will be appreciated that any number of micromachined 
elements may be operatively coupled to substrate 910 in, for example, a two 
dimensional array or other arrangement. Further, while Figure 9 illustrates an 
embodiment for which there is a one-to-one correspondence between 
micromachined light directing elements 930 and photodetector elements 905, for 
other embodiments, more than one photodetector element 930 may underlie a 
single micromachined element or vice versa. 
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For one embodiment, the micromachined light directing elements 930 are 
each individually controlled by micromachine control circuitry 912 to vary the 
angle of the light directing element 930 with respect to a surface of the substrate 
910. In this manner, the position of the opening 1007 in each of the 
5 micromachined light directing elements 930 is varied relative to the underlying 
photodetector(s) 905 and the light source(s) of interest 925. Varying the relative 
position of the openings 1007 in the light directing members 930 determines 
which point sources of light 925 and which combinations of point sources provide 
photons that are received at each individual photodetector element 905. 

10 For other embodiments, groups of light directing elements 930 may be 

controlled together to operate in a similar manner to each other. In this manner, 
the compound eye 900 may actually operate as multiple compound eyes to 
capture multiple individual images concurrently. 

While the angles of each of the micromachined light directing elements 

15 930a-d of Figure 9 vary in one of two directions, it will be appreciated that the 
compound eye 900 may include micromachined light directing elements that vary 
in any direction in response to control signals received from the micromachine 
control circuitry 912. 

The micromachined light directing elements 930 of one embodiment are 

20 . manufactured on the substrate 910 in accordance with techniques used to 
manufacture, for example, micro mirror switches and/or digital mirror devices. 
Examples of such techniques are described in, for example, a paper entitled, 
"Micromachined Adaptive Mirrors," by Gleb Vdovin of the Laboratory of 
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Electronic Instrumentation, Delft University of Technology. The paper can 
presently be found at http://guernsey.et.tudelft.nl/tyson4/. 

Such devices, however, typically do not include an opening such as the 
opening 1007. Thus, in addition to the known techniques for manufacturing 
5 micromachined elements, an additional processing action of providing an 
opening in the micromachined light directing element is provided. 

Techniques for designing micromachine control circuitry similar in many 
respects to the micromachine control circuitry 912 to individually control each of 

if' 15 

P the micromachined elements are also known to those of ordinary skill in the art 
{jj 10 and implemented in, for example, digital mirror devices manufactured by Texas 

J Instruments. The control circuitry 912 may differ in some respects from such 

111 

, circuitry, however, depending on the particular application for the compound eye 

m 900. 

H 

\ll Outputs of photodetector devices 905 are coupled to image processing 

Q 

H 1 5 circuitry 906 and/or to output circuitry 907 to provide a synthesized image 
corresponding to light source(s) 925. Image processing circuitry and output 
circuitry 907 may each provide capabilities similar to corresponding circuitry 
described in reference to other embodiments and may be configured in a similar 
manner. 

20 More specifically, depending on the application for the compound eye 

900, the image processing circuitry 906 may include a variety of different types of 
circuitry. For example, for some embodiments, such as where the micromachine 
control circuitry 912 controls the light directing elements 930 such that the 
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compound eye 900 actually operates as two distinct eyes, the image processing 
circuitry 906 may include edge enhancement circuitry, lateral inhibition circuitry, 
etc. For other embodiments, such as, for example, where it is desirable to be 
able to move any receptor's local visual field to any place in the entire visual field 
independently of other receptor fields, the image processing circuitry 906 and/or 
output circuitry 907 may be off-chip and may not include the above-described 
capabilities. 

In operation, light from light source(s) 925 is radiated towards the 
compound eye 900. Micromachine control circuitry 912 controls the angle of 
each of the micromachined light directing elements 930 with respect to a surface 
of the integrated circuit 910 by, in a manner well-known to those of skill in the 
micromachine arts. The particular angle of each of the micromachined light 
directing elements 930 determines from which point source(s) 925 photons are 
received at each photodetector element 905. 

Energy from photons received at each photodetector element 905 is 
converted to a corresponding electrical signal at outputs of each of the 
photodetector elements 905 which are interconnected and coupled with output 
circuitry 907 in such a way so as to provide an image corresponding to light 
source(s) 925 at an output. 

Micromachine control circuitry 912 may dynamically vary the angles of 
one or more of the micromachined light directing elements 930 in response to 
user input, a particular software program, feedback circuitry, a state machine or 
other types of input stimuli. The manner in which the light directing elements 930 
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are controlled may depend on the particular application for the compound eye 
900. For example, the compound eye 900 may provide a tracking mechanism to 
track movement of an object, however, other applications for compound eye 900 
will be appreciated by those of ordinary skill in the art. 

Figure 11 is a cross-sectional view of a compound eye 1 100 of another 
embodiment. The compound eye 1 100 includes micromachined light directing 
elements 1 105, each of which includes an integrated photodetector element. 
The light directing elements 1 105 are referred to herein as light scanning 
elements due to the fact that angle of the photodetector itself is adjusted by 
these elements to determine the point source(s) and/or angles from which light 
energy is received as described in more detail below. 

Figure 12 is an overhead view of such a micromachined light scanning 
element 1 105 and corresponding photodetector element 1205. The 
photodetector elements 1205 may be similar in construction, operation and/or 
other characteristics to photodetector elements of one or more of the other 
embodiments described herein. 

The compound eye 1 100 also includes micromachine control circuitry 
1112 and may include on-chip output circuitry 1 107. The micromachine control 
circuitry 1112 may be configured in a similar manner to the micromachine control 
circuitry 912 of Figure 9 to vary the angles of micromachined light scanning 
elements 1 105. Only a representative few light scanning elements 1 105a-f are 
shown in Figur 1 1 , each of which varies in only one of two directions. It will be 
appreciated, however, that the compound eye 1 100 may include many more 



25 




such micromachined light scanning elements, each of which may vary in 
directions relative to the substrate 1 1 10 other than those shown. 

The micromachined light scanning elements 1 105 may be manufactured 
in a manner similar to the light directing elements for the compound eye of 
5 Figure 9, with some modifications. For the micromachined light scanning 
elements 1 105, for example, an opening is not provided. Further, each of the 
micromachined light scanning elements 1 105 is fabricated to include a 
photodetector element 1205 as described above and to provide an output signal 
from the photodetector element to image processing circuitry 1 1 06 and/or output 

10 circuitry 1107. 

In operation, the angle of each of the micromachined light scanning 
elements 1 105 with integrated photodetector element 1205 is determined in 
response to micromachine control circuitry 1112. Micromachine control circuitry 
1112 may be responsive to user input, software instructions, state machine 

15 circuitry, other control circuitry, etc. to determine the angles of each of the 
integrated light scanning elements 1 105. 

The angle of each of the individual light scanning elements 1 105 
determines from which point source(s) 1 125 and at what angle light is received. 
For example, light from the point source 1 125a is blocked from being received by 

20 the photodetector on the scanning element 1 105c, but may be received by the 
remaining photodetectors on the scanning elements shown. Photons received 
by each of the photodetector elements 1205 cause a corresponding electrical 
signal to be produced at an output of the photodetector element 1205 that is 




provided to image processing circuitry 1 106 or to output circuitry 1 107, 
depending on the application. 

The image processing circuitry 1 106 and/or output circuitry 1 107 may be 
similar in configuration and/or operation to the corresponding circuitry described 
5 above in reference to Figure 9. The image processing circuitry 1 1 06 and/or 
1 107 integrates the output information from photodetector elements 1205 to 
synthesize one or more images captured by the eye 1 100. 

Either of the integrated circuit based compound eyes 900 or 1 100 may be 
used for a variety of unique applications as well as for other applications 

10 mentioned in reference to the compound eyes of other embodiments. For 
example, for one embodiment, one of the compound eyes 900 or 1 100 could 
dynamically be broken into multiple eyes, each capable of tracking an individual 
object for a given time period. This eye could then be dynamically reconfigured 
to operate as a single eye for another purpose. Other applications which may 

15 benefit from the capability to individually reposition the light scanning or directing 
elements and/or to capture multiple individual images simultaneously may also 
be a good target application for the compound eyes 900 and/or 1 100. 

Each of the above-described embodiments provides an integrated circuit- 
based compound eye including a synthetic aperture that is capable of being 

20 used for a variety of image sensing and/or processing applications. The 

integrated circuit-based compound eyes of various embodiments provide image 
sensing chips that do not require external lenses and thus, may provide a 
substantial cost savings over conventional image sensors with similar 
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capabilities. Further, because the compound eyes of various embodiments do 
not require an external lens, they are substantially planar and scale well to small 
form factors and weights. 

The applications for such integrated circuit-based compound eyes are 
5 innumerable. Video technologies, digital photography applications, graphics 
arts, video recognition, rugged optical sensors, etc. may all benefit from such an 
artificial compound eye. 

Other features and advantages of the integrated circuit-based compound 

P eye of various embodiments will be appreciated by those of ordinary skill in the 

p 

flJ 10 art. 

CO 

■j: Figure 13 is a flow diagram showing the image capture method of one 

m 

J* 5 embodiment. At block 1305, it is determined from which angles and/or which 

jffj point sources light energy is directed to associated photodetector elements. The 

\»± 

m photodetector elements produce an output that is related to the light energy 

w 

H 15 received at block 1310 and at block 1315, the outputs of the photodetector 
elements are integrated to synthesize an image associated with the point 
sources. 

It will be appreciated that the method of other embodiments may include 
additional actions not shown in Figure 13. 
20 Thus, various embodiments of an integrated circuit-based compound eye 

are described. In the foregoing specification, the invention has been described 
with reference to specific exemplary embodiments thereof. It will, however, be 
appreciated that various modifications and changes may be made thereto 
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without departing from the broader spirit and scope of the invention as set forth 
in the appended claims. The specification and drawings are, accordingly, to be 
regarded in an illustrative rather than a restrictive sense. 
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